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ABSTRACT 


Background: The screening for congenital hypothyroidism (CH) aims for early diagnosis and appropriate treatment. This 
programme reportedly did not achieve its quality assurance performance in terms of percentage of patients seen for 
confirmation and percentage of confirmed cases that received treatment within 14 days. Immediate attention is needed to 
assess the behaviour of the programme implementer. Therefore, the present study aims to develop and validate a 
questionnaire based on the Theory of Planned Behaviour (TPB) and the Attitude, Social Norms, and Self-Efficacy (ASE) 
Model. Methods: Relevant document analysis and extensive review of the articles were performed in the process of 
domain identification and item generation. The newly developed instrument was evaluated by six panel experts using 
Content Validity Index (CVI) and distributed to six respondents for face validity. A cross-sectional study was piloted among 
63 healthcare providers in a government hospital and two selected health clinics in Perak. Item analysis and reliability 
analysis were performed. Results: The initial item pool consisted of 45 items. The scale-level CVI (S-CVI) obtained based 
on the agreement of the panel experts was 0.96. As a result of the item analysis, eight items for the knowledge domain 
were dropped. As for the reliability analysis, one item for the attitude domain was dropped. In the end, the questionnaire 
consisted of 37 items. The Kuder-Richardson (KR20) value was 0.35, while Cronbach's alpha (CA) ranged from 0.720 to 
0.893. Conclusion: The newly developed questionaire has adequate validity and good internal reliability for quantifying 
providers’ attitude, perceived behaviour control (PBC), knowledge, and behaviour intention. 
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INTRODUCTION 


programme’ success. 

The programme’s performance is influenced by the 
providers, clients or patients, and programme support 
environment. A study conducted among healthcare 
providers in Asian countries revealed that government 
prioritisation, cooperation of healthcare providers as 


Congenital hypothyroidism (CH) refers to deficiency 
of thyroid hormone, which is relatively a common 
cause of mental retardation, developmental delay, and 
growth disturbance that is preventable.!'*! Globally, it 


is estimated that the incidence of the disease is one in 
2,000-4,000 live births.*! Studies have reported that 
in Malaysia, the incidence of CH occurs ranging from 
1:3,666 to 1:1,170 live births.'**! The screening for 
CH aims to ensure early diagnosis and appropriate 
treatment. However, in Malaysia, studies have 
reported that the programme did not achieve its 
quality assurance performance in terms of percentage 
of patients seen for confirmation and percentage of 
confirmed cases that received treatment within 14 
days.”:7! Therefore, there is a need to improve on the 
programme implementation activities to ensure the 
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well as public education and acceptance have 
contributed towards a_ successful screening 
programme."3! Before assessing others, it is important 
to assess the local healthcare providers in programme 
implementation. In view of this, immediate attention 
is needed to assess the behaviour of the programme 
implementers. 

Many studies have applied the Theory of Planned 
Behaviour (TPB),"**! and the Attitude, Social 
Norms, and Self Efficacy (ASE) Model in 
understanding the behavioural aspects of healthcare 
providers in programme implementation. Both models 
predict the occurrence of a specific behaviour,?°*! 
provided that the behaviour is intentional.!*!“*! TPB 
suggests that the intention of an individual to behave 
is determined by three main factors, namely attitude 
towards behaviour, subjective norms, and perceived 
behaviour control (PBC).4!“7] The ASE Model 
suggests that apart from the three determinants of 
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behavioural intention, factors such as knowledge, 
skills, and stimuli may play a role. As far as the 
researcher is concerned, there are no studies available 
pertaining to these in relation to the implementation of 
newborn screening programme. Most of the published 
studies are related to knowledge, attitude, and practice 
(KAP) in the context of newborn hearing screening 
programme."**! In view of this, there is a need to 
develop a new instrument in assessing the behaviour 
of healthcare providers focusing on __ the 
implementation of newborm screening for CH. 
Understanding these behaviours will be valuable in 
designing specific behaviour changing intervention 
strategies in order to improve the performance of the 
programme. 
In developing a new study instrument, rigorous 
development processes are expected to ensure it is 
reliable and valid.©?! This is essential to ensure the 
good quality of data that subsequently can be used in 
future research.' Thus, the purpose of this study is to 
develop and validate a questionnaire based on TPB 
and ASE Model in order to assess the attitude, PBC, 
knowledge, and behavioural intention of healthcare 
providers. 


MATERIALS AND METHODS 


Development of the instrument 
The development process involved several steps to 


ensure the newly developed instrument was valid 
and reliable. Relevant document analysis and 
extensive review of the articles on newborm 
screening,'®! congenital hypothyroidism screening 
programme, and studies related to TPB and ASE 
Model were performed so that all relevant domains 
and subdomains for each construct can be identified. 
In this study, attitude refers to workers’ feeling or 
opinion towards adherence to the implementation of 
the programme and its importance. PBC is defined 
as the overall evaluation of the degree to which 
workers believed that a screening programme for 
CH can be implemented as recommended; whereas 
behavioural intention refers to how likely workers 
intend to adhere to protocol in implementing the 
programme activities. For attitude, PBC, and 
behavioural intention, TPB was applied through a 
manual published specifically for health service 
researchers and existing studies, with domains or 
constructs based on TPB.!%! Based on the 
information gathered, two subdomains were 
identified for the attitude domain, which are attitude 
towards the implementation of programme activities 
and the importance of the screening programme. As 
for the PBC domain, direct measurement was used 
and this could be achieved by assessing self-efficacy 
and controllability of participants. Generalised 
intention measurement was used to develop items for 
the behavioural intention domain. 

For the knowledge domain, published studies 
evaluating healthcare providers in terms of 


knowledge in implementing screening programmes 
for CH are extremely limited. However, the 
researcher found numerous studies published on 
hearing screening programmes for newborns. After a 
detailed assessment, four articles were finally 
selected as a guide in developing the knowledge 
items.47°3°51 In addition, the existing documents 
related to the programme implementation were 
referred in this process.'©°! To facilitate the 
researcher in identifying the full content domain, a 
table of specifications was used. In this study, the 
knowledge domain refers to the score of knowledge 
items related to the screening programme, 
symptoms, and consequences of CH. Based on the 
information gathered, three subdomains were 
identified, which are knowledge on the screening 
programme, symptoms, and complications of the 
disease. After the identification of domains and 
subdomains was completed, item generation and 
instrument construction were performed. The initial 
item pool consisted of 45 items. 


Validation of the instrument 

The development process was followed by the 
validation process based on a study conducted by 
Farizan et al.!*! 


Content validation 

The newly developed instrument underwent a 
content validation process through a quantification 
process by a panel of experts (detailed review and 
assessment of CVI). In this process, a selected panel 
of experts evaluated the individual items as well as 
the entire instrument. The panel experts were 
identified among those who have vast experience, 
significant practical knowledge in this study field, 
and are up-to-date with the programme 
implementation activities. Each chosen expert was at 
least one of the following: (1) a Public Health 
Physician who is coordinating CH programmes; (2) 
a programme coordinator in a _ Paediatric 
Department; (3) a programme coordinator in a 
Pathology Department; (4) an academician in the 
public health related field (5) has published an article 
related to CH; and (6) a Family Medicine Specialist 
who has clinical expertise in the study field. Thus, a 
total of six experts were selected. The panel of 
experts selected was from district, state, and national 
levels. The panel consisted of two Public Health 
Physicians who were coordinating the programme 
(district and national levels), one Family Health 
senior lecturer from a public university, one Senior 
Paediatric Consultant from a state hospital, one 
Family Medicine Specialist inform a health clinic, 
and one Senior Science Officer from a laboratory of 
a state hospital. They were contacted by the 
researcher via a phone call and email to brief them 
regarding the study conceptualisations and item 
development for each construct. 
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A set of 45-item questionnaires was distributed to 
each expert. Specific instruction was provided 
regarding the review questionnaire. Definitions for 
content domain were provided and keywords that the 
experts needed to address were highlighted to 
emphasise major review points. The experts were 
instructed to evaluate in terms of item content 
(representativeness) and item style (clarity). 
Subsequently, the entire instrument was evaluated by 
the experts for comprehensiveness to allow the 
researcher to revise the instrument if the items did 
not represent the content domain or missing domain 
areas were identified. 
For the CVI assessment, the panel experts were 
required to evaluate each item using a 4-point scale 
intorduced by Davis.© : 1 = irrelevant, 2 = 
somewhat relevant, 3 = relevant and need correction, 
and 4 = very relevant. Subsequently, each item was 
categorised into two categories, namely irrelevant 
(items rated 1 and 2) and relevant (items rated 3 and 
4). Based on these two categories, the values of 
item-level Content Validity Index (I-CVI) and S- 
CVI were calculated. A minimum I-CVI of 0.83 is 
required if 6 expert panels are used (61). For S-CVI 
of a newly developed instrument, the minimum 
acceptable value is 0.8.!°71 


Face validation 

Once the content validity process was completed, the 
instrument was then distributed to six respondents 
involved in the programme implementation, but not 
included in the actual study, for face validity. This 
process was conducted to ensure the items developed 
were easy to understand. Sentence structure and 
grammer were also evaluated to ensure that the 
survey achieved the highest quality. Moreover, this 
process is very important to ascertain that the 
respondents are correct in their interpretation during 
the actual study. Based on the ratings and comments 
received from the respondents, the relevant items 
were reviewed. 


Pilot study 
A cross-sectional study was conducted among 


healthcare providers in a government hospital and 
two selected health clinics in the Perak state. The 
respondents comprised individuals who are involved 
in the programme implementation from receiving of 
samples until initiation of treatment. They were 
chosen in order to assess the validity and reliability 
of the newly developed instrument. For hospitals 
workers, they were from the Pathology and 
Paediatrics Departments. As for health clinics, it 
involved workers from the Maternal and Child 
Health (MCH) unit. Medical officers and nursing 
staff were recruited from the Paediatrics Department 
and health clinics, while for the Pathology 
Department, science officers and medical laboratory 
technicians (MLT) were recruited. Participants were 
selected using the purposive sampling method. Data 


collection was carried out from September to 
October 2018. A brief description and purpose of the 
study was given to the participants. Written consent 
was taken from the participants who agreed to 
participate. All participants had to complete a set of 
questionnaire individually and submit it to the 
researcher or focal person in the organisation on the 
same day. Out of 70 sets of questionnaires that were 
distributed, 63 workers had answered. Subsequently, 
item analysis and reliability analysis were performed 
based on the result of this pilot study. 


Difficulty Index and Discrimination Index 
For the knowledge domain, item analysis was 


performed to calculate the items’ difficulty index 
(DID and discriminantion index (RI). Both indexes 
ranged from 0 to 1.00. A smaller value of DI 
indicates a higher level of difficulty. Items with a DI 
value less than 0.2 need to be revised because they 
are considered too difficult. Meanwhile, items with 
an RI value less than 0.2 need to be revised as they 
cannot differentiate between knowledgeable and 
non-knowledgeable persons. '**! 


Reliabili 


For attitude, PBC, and behavioural intention 
domains, internal consistency reliability analysis and 
Cronbach's alpha (CA) values were applied. As the 
coefficient values are higher, the items in the 
questionnaire are considered more consistent to be 
used to measure the variables. The CA value of 0.65 
and above is accepted as correct and consistent (60). 
As for the knowledge domain, reliability was 
measured using the Kuder-Richardson Formula 20 
(KR20). KR20 values ranged from 0 to 1. High 
values indicate reliability, while values that are too 
high (greater than 0.90) are homogeneous."“*! 


Ethical consideration 

The approval for this study was obtained from the 
following: (1) Research and Ethics Committee, 
Medical Faculty, National University of Malaysia; 
(2) Medical Research and Ethics Committee 
(MREC), Ministry of Health Malaysia; (3) Deputy 
State Health Director (Public Health) of Perak; (4) 
Deputy State Health Director (Medical) of Perak; (5) 
respective Hospital Directors in Perak; and (6) 
respective District Health Officers in Perak. 


Data analysis 
Few analyses were involved in this study. For 


content validity, the values of I-CVI and S-CVI were 
calculated. Percentage was used in the face validity 
process. For the construct validation of the 
knowledge domain, calculations of DI and RI were 
involved. As for internal consistency, CA and KR-20 
values were calculated. 


RESULTS 
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Development of the instrument 
The initial pool of items related was compiled based 
on the extensive literature search of previous studies’ 
instruments. Apart from that, several items were 
adopted and modified to suit the study population. 
The initial item pool consisted of 45 items and were 
organised in an appropriate sequence. Items were 
prepared in a bilingual format (Malay and English). 
The attitude domain consisted of 15 items and 
ranged from (1) least important to (7) highly 
important. The PBC domain comprised six items and 
ranged from (1) strongly disagree to (7) strongly 
agree. As for the behavioural intention domain, it 
consisted of four items that measured compliance to 
timeframe, case management, guideline 
implementation, and cooperation and ranged from 
(1) least likely to (7) most likely. For the knowledge 
domain, there were 20 items with three responses, 
namely “Yes”, “No”, and “Unsure”, in which correct 
answers were given one point, while incorrect and 
“Unsure” answers were given zero points. 


Validation of the instrument 

Content validation 

All items for attitude, PBC, and behavioural 
intention domains had an I-CVI value of 1.0. 
However, some items were suggested for 
improvement in terms of words used and sequences. 
As a final draft, there were 15 items for the attitude 
domain, 6 items for the PBC domain, and 4 items for 
the behavioural intention domain. As for the 
knowledge domain, two items had an I-CVI value 
less than 0.83. Furthermore, there are three 
additional items suggested by the experts for further 
review. After the review, items with an I-CVI value 
less than 0.83 were dropped. All three additional 
items suggested were decided to be included in the 
content domain after being reviewed by the 
researchers. As a final draft, there were 21 items in 
the knowledge domain. 

Overall, based on the I-CVI value of all the items, 
the S-CVI value was found to be 0.96. As a final 
draft, all 46 items constructed for the four domains 
(15 items for the attitude domain, 6 items for the 
PBC domain, 4 items for the behavioural intention 
domain, and 21 items for the knowledge domain) 
were reviewed by the same expert panel in 14 days 
as recommended by Davis.[67] As a result, all 46 
proposed items were finalised and agreed by the 
experts as a result of the content validity process. 
Face validation 

From the six participants involved in rating the 
instruments, more than 83% agreed that the content, 
structure, and formatting of the questionnaire were 
relevant and easy to understand. A majority of 
participants also agreed that the writing style, font 
size, and spacing of the text were appropriate. Based 
on the ratings and comments received from the 
respondents, the relevant items were reviewed with 
no additional or deleted item. 


Difficulty Index and Discrimination Index 

None of the items had a DI value less than 0.2. Out 
of 21 items of the knowledge domain, 13 items had a 
DI value more than 0.9 and out these 13 items, four 
of them had a DI value of 1.0 and were considered 
very easy. These four items were reviewed to 
examine relevance and appropriateness and were 
finally dropped after discussions with the experts. 
Meawhile, the other nine items were retained. The 
remaining eight items (out of 21) with DI values 
ranging from 0.2 to 0.9 were also retained. As a 
result, 17 items were retained after the DI values 
were calculated. Based on these 17 items, nine items 
were found to have an RI value of less than 0.2 and 
were reviewed with the experts. As a result, four 
items were dropped because the RI value was too 
low. The other eight items with RI values within the 
range of 0.25-0.84 were retained. As a final result, 
13 items of knowledge domain were retained after 
the RI values were calculated. For the remaining 13 
items, the average value of DI was 0.86, which was 
very simple; whereas the average value of RI was 
0.27, which was moderate in distinguishing 
knowledgeable and less knowledgeable respondents. 


Reliabili 


For the attitude domain, out of the 15 items 
generated, the initial CA value was 0.68. After 
further discussion with the experts, the decision was 
made to drop one item with negative corrected item- 
total correlation in order to obtain a better CA value. 
The CA value of these 14 items was 0.893. For the 
PBC domain (six items) and behavioural intention 
domain (four items), the initial CA value was 0.78 
and 0.72, respectively. For the knowledge domain, 
based on the average DI and RI values of the 13 
items, the calculated KR20 value was 0.35. Finally, 
14 items for the attitude domain, 6 items for the PBC 
domain, 4 items for the behavioural intention 
domain, and 13 items for the knowledge domain 
were agreed to be included in this instrument. 


DISCUSSION 


The development process involved domain 
identification based on the TPB model and resulted 
with the construction of the instrument to ensure 
that the intrument was properly developed. With 
regard to the content validation process, the value 
of S-CVI obtained based on the agreement of the 
panel experts was 0.96, indicating excellent content 
validity as suggested by previous studies.!°7°! 
Additionally, based on the face validation process, 
the findings supported the present study’s content 
validation process as the instrument was agreed to 
be relevant, easy to understand, and appropriate by 
a majority of the participants. 

With regard to the pilot study conducted, validity 
of the knowledge domain was assessed using item 
analysis. The present study revealed that the 
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knowledge items were very simple and moderate in 
distinguishing knowledgeable and less 
knowledgeable respondents based on the value of 
average DI (0.86) and RI (0.27), as suggested in a 
previous study.'®! As for reliability analysis, the 
CA value for attitude, PBC, and behavioural 
intention domains indicated that the internal 
consistency was good. However, the knowledge 
domain’s value was found to be low (KR20=0.35), 
which may need improvement as suggested by a 
previous study.'! In this study, although the value 
of KR20 was less than 0.6, the items were agreed 
by the experts as comprehensive enough to 
measure knowledge. Finally, 13 items for the 
knowledge domain were retained. 

After rigorous development and _ validation 
processes, the new intrument demonstrated good 
reliability and excellent content validity in 
assessing the behaviour of healthcare providers. It 
is only apllicable for the Perak state or other similar 
settings in terms of the implementation of 
programme activities. However, this intrument can 
still be used as a guide for other settings that may 
have some diffrences in their implementation. As 
for other countries that may not have this screening 
programme as their priority yet, consideration to 
establish a very good and structured programme is 
suggested, and this intrument is valuable for future 
evaluation purposes. 

The involvement of the six panel experts in 
ensuring excellent content validity demonstrated 
that this intrument is comprehensive enough in 
determining the minimum requirement in terms of 
attitude, PBC, knowledge, and __ behavioural 
intention of healthcare providers in implementing 
the programme. Thus, strong attitude, high PBC, 
and knowledge score as well as full behavioural 
intention are required in order to implement the 
programme properly. 

In this study, a small sample size was found to be 
the limitation. Hence, item analysis and reliability 
analysis are suggested to be performed on the data 
of the actual study to strenghten the validity and 
reliability of the items. 


CONCLUSION 


The development process, content validity, face 
validity, item analysis, and reliability analysis have 
demonstrated adequate validity and good internal 
reliability for quantifying healthcare providers’ 
attitude, PBC, knowledge, and behavioural intention. 
Therefore, the survey instrument developed in this 
study is valuable in exploring and gaining better 
understanding in terms of the behaviour of 
healthcare providers in implementing the newborn 
screening programme for CH. 
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